AI Chatbots: A Conundrum for the FDA

Their names might infer ordinary folk, but these particular “ladies” lead existences far removed from the ordinary. 

They share a simple yet common purpose, and are pursuing that shared goal in a most unconventional way. 

Meet Molly, Ada, Florence, Clare, and Ginger—members not of a coffee clutch, but rather a new generation of artificial intelligence (AI) chatbots that are rapidly transforming healthcare accessibility, operational efficiency, and delivery.

“AI-based chatbots have emerged as promising tools for addressing longstanding challenges within the healthcare sector, particularly in improving access to information, patient support, and care delivery,” a PLOS Digital Health research article stated in February. “The development of AI-based chatbots is closely linked to advancements in artificial intelligence technologies.”

Those advancements date back six decades, starting with the original chatbot, ELIZA (another “female”), who could only match keywords and reply from a fixed script. Since then, however, chatbots have evolved to integrate natural language processing, machine learning, and multimodal interactions, achieving 80% to 90% accuracy rates. Most recently, generative AI has enabled chatbots to mimic users’ behaviors, understand and respond to questions, adapt their tone and language, and communicate in increasingly human-like ways.

This transformation from fixed scripts and simple rule-based systems to intelligent digital communication has driven the explosive growth and widespread adoption of healthcare AI chabots over the last several years.

Industry data projects the global healthcare chatbot market to grow 24% annually over the next four years to reach $4.4 billion, as AI, telehealth, and digital tools are further embedded in healthcare systems. 

Another driver of that growth is the rising number of chatbot users. Nearly one in three consumers (32%) reported consulting AI chatbots for health purposes last year, double the number from 2024, a Rock Health survey concluded. Moreover, 64% of AI users said they engage with these bots for health questions weekly or more often, according to survey results. 

While the growing use of AI chatbots has been welcomed by overworked physicians and clinicians, it has also raised concerns among medical professionals over their safety and lack of regulation.

Earlier this year, nonpartisan patient safety group ECRI chose the bots as the top health technology hazard of 2026, warning they can offer false or misleading information that could harm patients.

From product launches to M&A and manufacturing trends, the MPO Source delivers the medtech news you need twice a week. Subscribe here!

“Medicine is a fundamentally human endeavor. While chatbots are powerful tools, the algorithms cannot replace the expertise, education, and experience of medical professionals,” ECRI President/CEO Marcus Schabacker, M.D., Ph.D., noted. “Realizing AI’s promise while protecting people requires disciplined oversight, detailed guidelines, and a clear-eyed understanding of AI’s limitations.”

Indeed, the limitations of AI chatbots are becoming increasingly apparent, yet comprehensive guidelines governing their use remain largely lacking.

Notably, AI falls into a gray area within U.S. Food and Drug Administration (FDA) regulations. Under the agency’s rules, AI is considered a medical device and subject to regulation only if it is “intended for use” in diagnosing, curing, mitigating, treating, or preventing disease. AI used in administrative tasks (i.e., scheduling or billing) or general wellness information is not regulated unless its intended use falls within existing regulatory frameworks. 

An AI chatbot that transcribes notes, for example, would not be subject to FDA scrutiny but software that generates diagnostic or treatment suggestions could require agency oversight. 

The Digital Health Center of Excellence within FDA’s Center for Devices and Radiological Health is currently leading a discussion paper on “Considerations for the Regulation of Generative AI-Enabled Medical Devices.” However, it’s unclear whether that paper specifically addresses AI chatbots.

Not surprisingly, the statutory ambiguity surrounding AI chatbots has sparked significant debate among stakeholders.

On the one side are regulatory advocates like Quincy Martin and Alex Peile, who argue that AI chatbots like ChatGPT Health are medical devices and should be treated as such by the FDA.

“In January 2026, OpenAI launched ChatGPT Health. OpenAI promises ChatGPT Health will help users clarify medical concepts, understand test results, prepare for appointments, and explore healthcare and wellness strategies,” the pair wrote in an online posting in May (Martin is a 2025-2026 Petrie-Flom Center Student Fellow; Peile is a second-year Harvard Law School student). “Yet the product name is misleading: The marketing of the ‘health’ chatbot asserts the product is not a diagnostic or symptom-screeing tool. OpenAI plausibly sought to circumnavigate the definition of a medical device. If OpenAI wants to play doctor, it has to play by the rules.”

Pablo Garcia Quint disagrees. A technology policy analyst with the American Consumer Institute, Garcia Quint believes the FDA would be mistaken in applying medical device regulations to wellness chatbots. “The FDA labels a product or service a medical device if it intends to diagnose, cure, mitigate, treat, or prevent a specific disease, including mental health conditions. AI chatbots do none of this. Any FDA decision to treat AI mental health chatbots as medical devices would be a mistake.”